課程大綱

課程資訊

課程名稱

最佳控制
Optimal Control

開課學期

108-2

授課對象

電機資訊學院生醫電子與資訊學研究所

授課教師

張時中

課號

EE5051

課程識別碼

921 U2330

班次

學分

3.0

全/半年

半年

必/選修

選修

上課時間

星期二2,3,4(9:10~12:10)

上課地點

電二101

備註

總人數上限：30人

Ceiba 課程網頁

http://ceiba.ntu.edu.tw/1082EE5051_OC

課程簡介影片

核心能力關聯

核心能力與課程規劃關聯圖

課程大綱

為確保您我的權利,請尊重智慧財產權及不得非法影印

課程概述

1. Introduction and course overview
2. Unconstrained nonlinear optimization-line search, gradient and Newton methods;
3. Constrained nonlinear optimization-Lagrange multipliers.
4. Dynamic programming: principle of optimality, dynamic programming, discrete-tme LQR;
5. Dynamic programming in continuous time: Hamilton-Jacobi-Bellman equation, continuous-time LQR
6. Numerical solution in MATLAB and term project definition
7. Midterm exam
8. Calculus of variations
9. Calculus of variations applied to optimal control
10. Properties of optimal control solution
11. Constrained optimal control
12. Singular arcs
13. Estimators/Observers
14. Final exam
15. Term project presentation

課程目標

This course studies basic optimization and the principles of optimal control. It will be mainly on deterministic and some stochastic problems for both discrete and continuous systems. The course covers solution methods including numerical search algorithms, dynamic programming, variational calculus, and approaches based on Pontryagin's maximum principle. It will include examples and applications of multiple disciplines to motivate both theoretic study and solution algorithm design. Target students include those interested in the areas of optimization and control of various dynamic systems such as energy systems, robots, vehicles, communication networks, socioeconomic systems and biological systems.

課程要求

Linear Algebra, Differential Equations, Control Systems or Linear Systems, or consent of instructor.

預期每週課後學習時數

Office Hours

備註： TBD

指定閱讀

1. Principles of Optimal Control - MIT OpenCourseWare
https://ocw.mit.edu/courses/aeronautics-and-astronautics/16-323-principles-of-optimal-control-spring-2008/lecture-notes/
2. B.D.O. Anderson and J.B. Moore, Optimal Control Linear Quadratic Methods, Prentice Hall
3. F.L. Lewis and V.L. Syrmos, Optimal Control, John Wiley & Sons.
4. Prof. D. P. Bertsekas' Course Lecture Slides, 2015, http://www.athenasc.com/DP_Slides_2015.pdf

參考書目

評量方式
(僅供參考)

No.	項目	百分比	說明
1.	Classroom Participation	5%
2.	Homework	20%	Late homework will not be accepted except officially approved reasons.
3.	Term project	25%
4.	Mid Term	30%
5.	Final Exam	20%

課程進度

週次	日期	單元主題
第1週	03/03	Review of Dynamic Systems and Control Introduction to Optimization Typical Problems in Optimal Control and Estimation Reinforcement Learning and Optimal Control Course Overview and Syllabus Reading Assignment: Robert Stengel, “2018 Seminar Slides for Optimal Control and Estimation, seminar 1” available: http://www.stengel.mycpanel.princeton.edu/MAE546Seminar1.pdf H. J. Sussmann and J. C. Willems, “300 years of optimal control: From the Brachystochrone to the Maximum Principle,” IEEE Control Systems, 1997. Chapter 1, F.L. Lewis and V.L. Syrmos, Optimal Control, http://www.uta.edu/utari/acs/FL books/Lewis optimal control 3rd edition 2012.pdf
第2週	03/10	Unconstrained Nonlinear Optimization Convexity Line search Gradient method Newton method Reading Assignments: 1. J. P. How, Principles of Optimal Control - MIT OpenCourseWare Lecture note - Unit 1 2. Chapter 7 and Chapter 8 (Sections 8.1-8.8), Linear and Nonlinear Programming Programming. Third Edition. David G. Luenberger. Stanford University. Yinyu Ye. Stanford University https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/Linear%20and%20Nonlinear%20Programming.pdf
第3週	03/17	Unconstrained Nonlinear Optimization Line search: Armijo rule Optimality conditions N-dimension problems Gradient method Newton method Constrained Optimization Optimality conditions Lagrange multipliers Reading Assignments: 1. J. P. How, Principles of Optimal Control - MIT OpenCourseWare Lecture note - Unit 1 2. Chapter 7 and Chapter 8 (Sections 8.1-8.8), Linear and Nonlinear Programming Programming. Third Edition. David G. Luenberger. Stanford University. Yinyu Ye. Stanford University https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/Linear%20and%20Nonlinear%20Programming.pdf
第4週	03/24	Constrained Optimization (Cont.) Optimality conditions Lagrange multipliers Dynamic Programming – Introduction Discrete time system: DP equation and example Continuous time system: DP equation and example Reading Assignments: 1. Chapter 11, Linear and Nonlinear Programming Programming. Third Edition. David G. Luenberger. Stanford University. Yinyu Ye. Stanford University https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/Linear%20and%20Nonlinear%20Programming.pdf 2. Lec 2 , J. P. How, Principles of Optimal Control - MIT OpenCourseWare Lecture note 3. Chapter 2 , F.L. Lewis and V.L. Syrmos, Optimal Control.
第5週	03/31	Duality in Nonlinear Convex Programming Conjugate functions Dual convex program Duality and Optimality Dynamic Programming Principle of optimality Dynamic programming – Discrete time Discrete Time LQR and DP Solution Reading Assignments: Chapter 5, “Duality in Nonlinear Convex Programming,” Nonlinear Programming-Analysis and Methods, M. Avriel, Prentic-Hall, 1976. EE363: Lecture 1 Slides Professor Stephen Boyd, Stanford University, https://stanford.edu/class/ee363/lectures/dlqr.pdf F.L. Lewis, D. Vrabie, and V.L. Syrmos, Optimal Control Chapter 6. J. P. How, Principles of Optimal Control - MIT OpenCourseWare Lecture note – Lec 3 Prof. Bertsekas' Course Lecture Slides, 2015 Lecture 2 and Lecture 3 (pages 15~36/302)
第6週	04/07	Dual Method Discrete Time LQR and DP Solution-Finite Horizon LQR cost function multi-objective interpretation LQR via least-squares dynamic programming solution steady-state LQR control extensions: time-varying systems, tracking problems LQR via Lagrange Multipliers useful matrix identities linearly constrained optimization Reading Assignments: EE363: Lecture 1 Slides Professor Stephen Boyd, Stanford University, https://stanford.edu/class/ee363/lectures/dlqr.pdf EE363: Lecture 2 Slides Professor Stephen Boyd, Stanford University, https://stanford.edu/class/ee363/lectures/lqr-lagrange.pdf F.L. Lewis, D. Vrabie, and V.L. Syrmos, Optimal Control Chapter 2.
第7週	04/14	Discrete Time LQR and DP Solution-Finite Horizon dynamic programming solution (Cont.) steady-state LQR control extensions: time-varying systems, tracking problems LQR via Lagrange Multipliers useful matrix identities linearly constrained optimization Infinite Horizon LQR via Lagrange Multipliers infinite horizon LQR problem dynamic programming solution receding horizon LQR control closed-loop system Reading Assignments: EE363: Lecture 1, 2 & 3 Slides Professor Stephen Boyd, Stanford University, https://stanford.edu/class/ee363/lectures/dlqr.pdf https://stanford.edu/class/ee363/lectures/lqr-lagrange.pdf https://stanford.edu/class/ee363/lectures/dlqr-ss.pdf F.L. Lewis, D. Vrabie, and V.L. Syrmos, Optimal Control Chapter 2.
第8週	04/21	Differential Dynamic Programming [1,2] Introduction Applications to 大甲溪、濁水溪水力發電調度 Continuous Time LQR Continuous-time LQR problem Dynamic programming solution Hamiltonian system and two point boundary value problem Infinite horizon LQR Direct solution of ARE via Hamiltonian Reading Assignments: S. Yakowitz and B. Rutherford, “Computational Aspects of Discrete-Time Optimal Control,” Applied Mathematica and Computation, 15:29-45, 1984. S. C. Chang, C. H. Chen, I. K. Fong, P. B. Luh, ”Hydroelectric Generation Scheduling with an Effective Differential Dyanmic Programming Algorithm,” IEEE Trans. on Power System Engienering, Aug. 1990. EE363: Lecture 4 Slides Professor Stephen Boyd, Stanford University, https://stanford.edu/class/ee363/lectures/clqr.pdf
第9週	04/28	Continuous Time LQR Hamiltonian system and two point boundary value problem (Cont.) Infinite horizon LQR Direct solution of ARE via Hamiltonian Discrete Time Stochastic Optimal Control Linear-quadratic stochastic control problem Solution via dynamic programming Introduction to Stochastic Dynamic Programming The basic problem, Principle of optimality DP example: Stochastic problem The general DP algorithm State augmentation Reading Assignments: EE363: Lecture 4 Slides Professor Stephen Boyd, Stanford University, https://stanford.edu/class/ee363/lectures/clqr.pdf EE363: Lecture 5 Slides Professor Stephen Boyd, Stanford University, https://stanford.edu/class/ee363/lectures/stoch_lqr.pdf Dimitri P. Bertsekas, Lecture Slides on Dynamic Programming, MIT http://www.mit.edu/~dimitrib/DP_Slides_2015.pdf
第10週	5/5	Midterm Exam (Closed Book) Scope: Week1 - Week8 subjects.
第11週	05/12	Discrete Time Stochastic Optimal Control (Cont.) Linear-quadratic stochastic control problem Solution via dynamic programming Introduction to Stochastic Dynamic Programming The basic problem, Principle of optimality DP example: Stochastic problem The general DP algorithm State augmentation Reading Assignments: EE363: Lecture 5 Slides Professor Stephen Boyd, Stanford University, https://stanford.edu/class/ee363/lectures/stoch_lqr.pdf Dimitri P. Bertsekas, selected parts of Lecture 2 ~5 Slides on Dynamic Programming, MIT http://www.mit.edu/~dimitrib/DP_Slides_2015.pdf
第12週	05/19	Review of Continuous Time Optimal Control Basic Problem of Calculus of Variations (CoV) CoV: Fixed terminal condition Euler-Lagrange Equation CoV: Free terminal conditions CoV Examples Term Project Proposal Presentation Corner Constraint & Terminal State Constraint Reading Assignments: J. P. How, Principles of Optimal Control - MIT OpenCourseWare Lecture note – Lec 5 D. Kirk, Optimal Control Theory- An introduction, Chapter 4 D. Liberzon, Calculus of Variations and Optimal Control Theory, Chapter 2 D. S. Naidu, Optimal Control Systems, Chapter 2
第13週	05/26	CoV: Fixed terminal condition Euler-Lagrange Equation CoV: Free terminal conditions CoV Examples Corner Constraint & Terminal State Constraint Simple Optimal Control Problem Hamiltonian & necessary conditions of OC Reading Assignments: J. P. How, Principles of Optimal Control - MIT OpenCourseWare Lecture note – Lecs 5 ,6 D. Kirk, Optimal Control Theory- An introduction, Chapter 4 D. Liberzon, Calculus of Variations and Optimal Control Theory, Chapter 2 D. S. Naidu, Optimal Control Systems, Chapter 2
第14週	06/02	Corner Constraint & Terminal State Constraint Simple Optimal Control Problem Hamiltonian & necessary conditions of OC Linear Quadratic Optimal Control Zermelo’s problem Reading Assignments: J. P. How, Principles of Optimal Control - MIT OpenCourseWare Lecture note – Lec 5, 6 (Slides adopted from How’s notes) D. Kirk, Optimal Control Theory- An introduction, Chapter 4 D. Liberzon, Calculus of Variations and Optimal Control Theory, Chapter 3 D. S. Naidu, Optimal Control Systems, Chapter 4
第15週	06/09	Linear Quadratic Optimal Control Zermelo’s Problem (Skip) Optimal Control with Point State Constraints Optimal Control with Input Constraints Pontryagin’s Maximum/Minimum Principle (PMP) Application of PMP: Minimum Time Example Reading Assignments: J. P. How, Principles of Optimal Control - MIT OpenCourseWare Lecture note – Lec 8 & 9 D. Kirk, Optimal Control Theory- An introduction, Chapter 5 D. Liberzon, Calculus of Variations and Optimal Control Theory, Chapter 4 D. S. Naidu, Optimal Control Systems, Chapter 6
第16週	06/16	Final Exam Scope: 1. Subjects of mid-term exam 30% 2. Simple LQG Optimal Control, Calculus of Variation and Optimal Control 70% (Lecture Notes 10 - 14 and Homework Problems).
第17週	06/23	Submit to COOL slides before 6/23/2020) Presentation: 6/23/2020 Final report due: by the end of 6/26/2020 Please be aware of the plagiarism issue and give suitable citations and references.